Regret Analysis of a Markov Policy Gradient Algorithm for Multiarm Bandits

We consider a policy gradient algorithm applied to finite-arm bandit problem with Bernoulli rewards. allow learning rates depend on the current state of rather than using deterministic time-decreasing rate. The forms Markov chain probability simplex. apply Foster–Lyapunov techniques analyze stability this chain. prove that, if are well-chosen, then is transient chain, and converges optimal arm logarithmic or polylogarithmic regret.